Method for analyzing an image of a dental arch

ABSTRACT

A method for analyzing an image, called “analysis image”, of a dental arch of a patient. The method in which the analysis image is submitted to a neural network, in order to determine at least one value of an image attribute relating to the analysis image. The analysis image is a photograph or an image taken from a film. The image attribute relates to a position or an orientation or a calibration of an acquisition apparatus used to acquire the analysis image or a combination thereof, or a quality of the analysis image, and in particular relating to the brightness, to the contrast or to the sharpness of the analysis image, or a combination thereof.

RELATED APPLICATION

Benefit is claimed from U.S. application Ser. No. 16/031,125 filed onJul. 10, 2018, the entire disclosure of which is incorporated herein byreference.

TECHNICAL FIELD

The present invention relates to the field of dental arch imageanalysis.

STATE OF THE ART

The most recent orthodontic treatments use images to assess thetherapeutic situations. This assessment is conventionally performed byan orthodontist, which requires the patient to transmit these images tothe orthodontist, even to make an appointment.

There is an ongoing need for a method simplifying the analysis of theimages of dental arches of patients.

One aim of the invention is to address this need.

SUMMARY OF THE INVENTION

The invention proposes a method for analyzing an image, called “analysisimage”, of a dental arch of a patient, a method in which the analysisimage is submitted to a deep learning device, preferably a neuralnetwork, in order to determine at least one value of a tooth attributerelating to a tooth represented on the analysis image, and/or at leastone value of an image attribute relating to the analysis image.

Analysis by Tooth

The invention proposes in particular a method for detailed analysis ofan image called “analysis image” of a dental arch of a patient, saidmethod comprising the following steps:

-   -   1) creation of a learning base comprising more than 1000 images        of dental arches, or “historical images”, each historical image        comprising one or more zones each representing a tooth, or        “historical tooth zones”, to each of which, for at least one        tooth attribute, a tooth attribute value is assigned;    -   2) training of at least one deep learning device, preferably a        neural network, by means of the learning base;    -   3) submission of the analysis image to said at least one deep        learning device for it to determine at least one probability        relating to an attribute value of at least one tooth represented        on a zone representing, at least partially, said tooth in the        analysis image, or “analysis tooth zone”;    -   4) determination, as a function of said probability, of the        presence of a tooth of said arch at a position represented by        said analysis tooth zone, and of the attribute value of said        tooth.

A first deep learning device, preferably a neural network, may inparticular be implemented to assess a probability relating to thepresence, at a location of said analysis image, of an analysis toothzone.

A second deep learning device, preferably a neural network, may inparticular be implemented to assess a probability relating to the typeof tooth represented in an analysis tooth zone.

As will be seen in more detail hereinafter in the description, adetailed analysis method according to the invention advantageously makesit possible to immediately recognize the content of the analysis image.

The analysis image may advantageously be classified automatically. Itmay also be used immediately by a computer program.

The invention relies on the use of a deep learning device, preferably aneural network, the performance of which is directly linked to therichness of the learning base. There is therefore also a need for amethod that makes it possible to rapidly enrich the learning base.

The invention therefore relates also to a method for enriching alearning base, notably intended for the implementation of a detailedanalysis method according to the invention, said enrichment methodcomprising the following steps:

-   -   A) at an “updated” instant, production of a model of a dental        arch of a patient, or “updated reference model”, and        segmentation of the updated reference model so as to produce,        for each tooth, a “tooth model”, and, for at least one tooth        attribute, assignment of a tooth attribute value to each tooth        model;    -   B) preferably less than 6 months, preferably less than 2 months,        preferably less than 1 month, preferably less than a fortnight,        preferably less than 1 week, preferably less than 1 day before        or after the updated instant, preferably substantially at the        updated instant, acquisition of at least one, preferably at        least three, preferably at least ten, preferably at least one        hundred images of said arch, or “updated images” in respective        real acquisition conditions;    -   C) for each updated image, search for virtual acquisition        conditions suitable for an acquisition of an image of the        updated reference model, called “reference image”, exhibiting a        maximum match with the updated image in said virtual acquisition        conditions, and acquisition of said reference image;    -   D) identification, in the reference image, of at least one zone        representing a tooth model, or “reference tooth zone”, and, by        comparison of the updated image and of the reference image,        determination, in the updated image, of a zone representing said        tooth model, or “updated tooth zone”;    -   E) assignment, to said updated tooth zone, of the tooth        attribute value or values of said tooth model;    -   F) addition of the updated image enriched with a description of        said updated tooth zone and its tooth attribute value or values,        or “historical image”, in the learning base.

In particular, each execution of the method described in WO 2016/066651preferably generates more than three, more than ten, preferably morethan one hundred updated images which, by an automated processing bymeans of the updated reference model, may produce as many historicalimages.

In a particular embodiment, the method for enriching a learning basecomprises, in place of the steps A) to C), the following steps:

-   -   A′) at an initial instant, production of a model of a dental        arch of a patient, or “initial reference model”, and        segmentation of the initial reference model so as to produce,        for each tooth, a “tooth model”, and, for at least one tooth        attribute, assignment of a tooth attribute value to each tooth        model;    -   B′) at an updated instant, for example spaced apart by more than        a fortnight, preferably more than a month, even more than two        months from the initial instant, acquisition of at least one,        preferably at least three, preferably at least ten, preferably        at least one hundred images of said arch, or “updated images”,        in respective real acquisition conditions;    -   C′) for each updated image, search, by deformation of the        initial reference model, for an updated reference model and        virtual acquisition conditions suitable for an acquisition of an        image of the updated reference model, called “reference image”,        exhibiting a maximum match with the updated image in said        virtual acquisition conditions.

This method advantageously makes it possible, after generation of theinitial reference model, preferably by means of a scanner, to enrich thelearning base at different updated instants, without it being necessaryto perform a new scan, and therefore without the patient having to go tothe orthodontist. He or she may in fact acquire the updated images himor herself, as described in WO 2016/066651.

A single orthodontic treatment may thus lead to the production ofhundreds of historical images.

The invention relates also to a method for training a deep learningdevice, preferably a neural network, comprising an enrichment of alearning base according to the invention, then the use of said learningbase to train the deep learning device.

Global Analysis

The detailed analysis method described above advantageously allows for afine analysis of the analysis image, the situation of each beingpreferably assessed.

Alternatively, the deep learning device may be used globally, thelearning base containing historical images whose description provides aglobal attribute value for the image. In other words, the value of theimage attribute relates to the whole image and not to part of the image.The attribute is not then a “tooth” attribute, but is an “image”attribute. For example, this image attribute may define whether, inlight of the image as a whole or of a part of the image, the dentalsituation “is pathological” or “is not pathological”, without each toothbeing examined The image attribute also makes it possible to detect, forexample, whether the mouth is open or closed, or, more generally,whether the image is suitable for a subsequent treatment, for examplewhether it makes it possible to monitor the occlusion.

The image attribute may in particular relate to

-   -   a position and/or an orientation and/or a calibration of an        acquisition apparatus used to acquire said analysis image,        and/or    -   a quality of the analysis image, and in particular relating to        the brightness, to the contrast or to the sharpness of the        analysis image, and/or    -   the content of the analysis image, for example to the        representation of the arches, of the tongue, of the mouth, of        the lips, of the jaws, of the gums, of one or more teeth or of a        dental, preferably orthodontic, appliance.

When the image attribute refers to the content of the image, thedescription of the historical images of the learning base specifies acharacteristic of this content. For example, it may specify the positionof the tongue (for example “retracted”) or the opening of the mouth ofthe patient (for example mouth open or closed) or the presence of arepresentation of a dental, preferably orthodontic, appliance and/or ofits condition (for example appliance intact, broken or damaged).

A tooth attribute value may be used to define a value for an imageattribute. For example, if a value of a tooth attribute is “toothdecayed”, the value of an image attribute may be “unsatisfactory dentalsituation”.

The image attribute may in particular relate to a therapeutic situation.

The invention proposes a method for global analysis of an analysis imageof a dental arch of a patient, said method comprising the followingsteps:

-   -   1′) creation of a learning base comprising more than 1000 images        of dental arches, or “historical images”, each historical image        comprising an attribute value for at least one image attribute,        or “image attribute value”;    -   2′) training of at least one deep learning device, preferably a        neural network, by means of the learning base;    -   3′) submission of the analysis image to the deep learning device        for it to determine, for said analysis image, at least one        probability relating to said image attribute value, and        determination, as a function of said probability, of a value for        said image attribute for the analysis image.

The image attribute may in particular relate to the orientation of theacquisition apparatus upon the acquisition of the analysis image. It mayfor example take the values “front photo”, “left photo” and “rightphoto”.

The image attribute may also relate to the quality of the image. It mayfor example take the values “insufficient contrast” and “acceptablecontrast”.

The image attribute may also relate to the dental situation of thepatient, for example relate to the presence of decay or to the conditionof a dental, preferably orthodontic, appliance worn by the patient(“degraded” or “in good condition” for example) or to the suitability ofthe dental, preferably orthodontic, appliance to the treatment of thepatient (for example “unsuitable” or “suitable”).

The image attribute may also relate to the “presence” or “absence” of adental, preferably orthodontic, appliance, or to the state of opening ofthe mouth (“mouth open”, “mouth closed” for example).

As will be seen in more detail hereinafter in the description, a globalimage analysis method according to the invention advantageously makes itpossible to immediately and globally assess the content of the analysisimage. In particular, it is possible to globally assess a dentalsituation and, for example, deduce therefrom the need to consult anorthodontist.

Definitions

A “patient” is a person for whom a method according to the invention isimplemented, independently of whether or not this person is following anorthodontic treatment.

“Orthodontist” should be understood to mean any person qualified toprovide dental care services, which also includes a dentist.

A “dental part”, in particular “orthodontic part”, should be understoodto mean all or part of a dental, in particular orthodontic, appliance.

An orthodontic part may in particular be an orthodontic aligner. Such analigner extends in such a way as to follow the successive teeth of thearch on which it is fixed. It defines a generally “U” shaped channel,the shape of which is determined to ensure the fixing of the aligneronto the teeth, but also as a function of a desired target positioningfor the teeth. More specifically, the shape is determined in such a waythat, when the aligner is in its service position, it exerts stressestending to move the teeth being treated to their target positioning, orto hold the teeth in this target positioning.

The “service position” is the position in which the dental ororthodontic part is worn by the patient.

“Model” should be understood to be a digital three-dimensional model. Anarrangement of tooth models is therefore a model.

“Image” should be understood to be an image in two dimensions, like aphotograph or an image taken from a film. An image is formed by pixels.

A “reference image” is a view of a “reference” model.

“Image of an arch” or “model of an arch” should be understood to be arepresentation of all or part of said arch. Such a representation ispreferably in colors.

The “acquisition conditions” of an image specify position and theorientation in space of an acquisition apparatus acquiring this image inrelation to the teeth of the patient (real acquisition conditions) or toa model of teeth of the patient (virtual acquisition conditions), andpreferably the calibration of this acquisition apparatus. Acquisitionconditions are said to be “virtual” when they correspond to a simulationin which the acquisition apparatus would be in said acquisitionconditions (theoretical positioning and preferably calibration of theacquisition apparatus) relative to a model.

In virtual conditions of acquisition of a reference image, theacquisition apparatus may be also qualified as “virtual”. The referenceimage is in fact acquired by a hypothetical acquisition apparatus,having the characteristics of the “real” acquisition apparatus used forthe acquisition of the real images, and in particular of the updatedimages.

The “calibration” of an acquisition apparatus is made up of all thevalues of the calibration parameters. A “calibration parameter” is aparameter intrinsic to the acquisition apparatus (unlike its positionand orientation) whose value influences the image acquired. Preferably,the calibration parameters are chosen from the group formed by thediaphragm aperture, the exposure time, the focal distance and thesensitivity.

A “discriminating information item” is a characteristic feature whichmay be extracted from an image (“image feature”), conventionally bycomputer processing of this image.

A discriminating information item may exhibit a variable number ofvalues. For example, a contour feature may be equal to 1 or 0 dependingon whether a pixel belongs or does not belong to a contour. A brightnessfeature may take a large number of values. The processing of the imagemakes it possible to extract and quantify the discriminating informationitem.

The discriminating information item may be represented in the form of a“map”. A map is thus the result of a processing of an image in order toreveal the discriminating information item, for example the contour ofthe teeth and of the gums.

“Match” or “fit” between two objects designates a measurement of thedifference between these two objects. A match is maximal (“best fit”)when it results from an optimization making it possible to minimize saiddifference.

An object modified to obtain a maximum match may be qualified as“optimal” object.

Two images or “views” which exhibit a maximal match representsubstantially at least one same tooth, in the same way. In other words,the representations of the tooth on these two images may besubstantially superimposed.

The search for a reference image exhibiting a maximal match with anupdated image is performed by searching for the virtual acquisitionconditions of the reference image exhibiting a maximal match with thereal acquisition conditions of the updated image.

By extension, a model exhibits a maximal match with an image when thismodel has been chosen from several models because it allows a viewexhibiting a maximal match with said image and/or when this image hasbeen chosen from several images because it exhibits a maximal match witha view of said model.

In particular, an updated image is in maximal match with a referencemodel when a view of this reference model provides a reference imageshowing a maximal match with the updated image.

The comparison between two images results preferably from the comparisonof two corresponding maps. “Distance” is used conventionally to denote ameasurement of the difference between two maps or between two images.

The “metaheuristic” methods are known optimization methods. They arepreferably chosen from the group formed by

-   -   evolutionary algorithms, preferably chosen from: evolution        strategies, genetic algorithms, differential evolution        algorithms, distribution estimation algorithms, artificial        immunity systems, shuffled complex evolution path recomposition,        simulated annealing, ant colony algorithms, particular swarm        optimization algorithms, search with taboos, and the GRASP        method;    -   the kangaroo algorithm,    -   the Fletcher and Powell method,    -   the sound effects method,    -   stochastic tunneling,    -   random-restart hill climbing,    -   the cross-entropy method, and    -   the hybrid methods between the abovementioned metaheuristic        methods.

“Description” of an image denotes information particularly relating tothe definition of the tooth zones of this image and to the toothattribute values which are associated with them, and/or relating to animage attribute value of said image. The number of possible values for atooth attribute or an image attribute is unlimited.

A “historical” image is an image of a dental arch enriched with adescription. The tooth zones of a historical image are qualified as“historical tooth zones”.

“Comprising” or “including” or “exhibiting” should be interpretedwithout restriction, unless indicated otherwise.

BRIEF DESCRIPTION OF THE FIGURES

Other features and advantages of the invention will become more apparenton reading the following detailed description and studying the attacheddrawing in which:

-   -   FIG. 1 represents, schematically, the different steps of a        method for detailed analysis of an image, according to the        invention;    -   FIG. 2 represents, schematically, the different steps of a        method for enriching a learning base, according to the        invention;    -   FIG. 3 represents, schematically, the different steps of a        variant of a method for enriching a learning base, according to        the invention;    -   FIG. 4 represents, schematically, the different steps of a        method for global analysis of an image, according to the        invention;    -   FIG. 5 represents, schematically, the different steps of a        step C) of a method for enriching a learning base, according to        the invention;    -   FIGS. 6 and 18 represent, schematically, the different steps of        a method for modeling the dental arch of a patient, according to        the invention;    -   FIG. 7 represents, schematically, the different steps of a        method for assessing a dental situation of a patient, according        to the invention;    -   FIG. 8 represents, schematically, the different steps of a        method for acquiring an image of a dental arch of a patient,        according to the invention;    -   FIG. 9 represents, schematically, the different steps of a        method for assessing the shape of an orthodontic aligner of a        patient, according to the invention;    -   FIG. 10 represents an example of a reference image of an initial        reference model;    -   FIG. 11 (11A-11D) illustrates a way of processing to determine        the tooth models in an initial reference model, as described in        WO 2016 066651, with FIG. 11A representing a reference image,        FIG. 11B representing a corresponding reference map, FIG. 11C        representing a map relating to the contour of the teeth, and        FIG. 11D representing a segmented reference model;    -   FIG. 12 (12A-12D) illustrates the acquisition of an image by        means of a retractor, an operation of cropping of this image,        and the processing of an updated image making it possible to        determine the contour of the teeth, as described in WO 2016        066651;    -   FIG. 13 schematically illustrates the relative position of        registration marks 12 of a retractor 10 on updated images 14 ₁        and 14 ₂, according to the directions of observation represented        by broken lines;    -   FIGS. 14 and 15 represent an orthodontic aligner, in perspective        and plan views, respectively;    -   FIG. 16 illustrates the step e) described in WO 2016 066651;    -   FIG. 17 (17A-17C) illustrates an enrichment method according to        the invention, with FIG. 17A showing an example of updated        image, FIG. 17B showing the reference image exhibiting a maximal        match with the updated image, and FIG. 17C illustrating the        transfer of the tooth numbers to the updated tooth zones; and    -   FIG. 18 is described above with FIG. 6.

DETAILED DESCRIPTION

A detailed analysis method according to the invention requires thecreation of a learning base. This creation preferably implements amethod comprising the steps A) to F), or, in one embodiment, in place ofthe steps A) to C), preferably the steps A′) to C′).

First Main Embodiment of the Enrichment Method

The step A) is intended for the production of an updated reference modelmodeling an arch of the patient. It preferably comprises one or more ofthe features of the step a) of WO 2016 066651, incorporated byreference, to produce an initial reference model. The updated referencemodel is preferably created with a 3D scanner. Such a so-called “3D”model may be observed from any angle. An observation of the model, froma determined angle and distance, is called a “view” or “referenceimage”.

FIG. 11A is an example of reference image.

The updated reference model may be prepared from measurements performedon the teeth of the patient or on a molding of his or her teeth, forexample a plaster molding.

For each tooth, from the updated reference model, a model of said tooth,or “tooth model”, is defined (FIG. 11D). This operation, known per se,is called “segmentation” of the updated reference model.

In the updated reference model, a tooth model is preferably delimited bya gingival margin which may be broken down into an interior gingivalmargin (of the side of the interior of the mouth relative to the tooth),an exterior gingival margin (oriented toward the outside of the mouthrelative to the tooth) and two lateral gingival margins.

One or more tooth attributes are associated with the tooth models as afunction of the teeth that they model.

The tooth attribute is preferably chosen from a tooth number, a toothtype, a tooth shape parameter, for example a tooth width, in particulara mesio-palatine width, a thickness, a crown height, an index of mesialand distal deflection of the incisive margin, or a level of abrasion, atooth appearance parameter, in particular a translucency index or acolor parameter, a parameter relating to the condition of the tooth, forexample “abraded”, “broken”, “decayed” or “fitted” (that is to say incontact with a dental, preferably orthodontic, appliance), an age forthe patient, or a combination of these attributes. A tooth attribute ispreferably an attribute which only relates to the tooth modeled by thetooth model.

A tooth attribute may be a relative position of a tooth, in particular arelative position of a tooth with respect to another tooth, preferablyan adjacent tooth, to an orthodontic aligner, or to an archwire, or to adental arch, in particular an ideal dental arch, or to the center orbarycenter of a tooth model, or to the center or barycenter of a dentalarch model.

A tooth attribute may be an angle relative to a tooth, in particularrelative to another tooth, preferably an adjacent tooth, to anorthodontic aligner, or to an archwire, or to a dental arch, inparticular an ideal dental arch.

A tooth attribute may be a distance relative to a tooth, in particularrelative to another tooth, preferably an adjacent tooth, to anorthodontic aligner, or to an archwire, or to a dental arch, inparticular an ideal dental arch, or to the center or barycenter of atooth model, or to the center or barycenter of a dental arch model.

A tooth attribute value may be assigned to each tooth attribute of aparticular tooth model.

For example, the “tooth type” tooth attribute will have the value“incisor”, “canine” or “molar” depending on whether the tooth model isthat of an incisor, of a canine or of a molar, respectively.

The assignment of the tooth attribute values to the tooth models may bemanual or, at least partly, automatic. For example, if the value of atooth attribute is identical whatever the tooth model, such as for the“age of the patient” tooth attribute, it may be sufficient to assign avalue to one tooth model to determine the value of this attribute forthe other tooth models.

Similarly, the tooth numbers are conventionally assigned according to astandard rule. It is therefore sufficient to know this rule and thenumber of a tooth modeled by a tooth model to calculate the numbers ofthe other tooth models.

In a preferred embodiment, the shape of a particular tooth model isanalyzed so as to define its tooth attribute value, for example itsnumber. This shape recognition is preferably performed by means of adeep learning device, preferably a neural network. Preferably, a libraryof historical tooth models is created, each historical tooth modelhaving a value for the tooth attribute, as described hereinbelow (stepa)), the deep learning device is trained with views of the historicaltooth models of this library, then one or more views of the particulartooth model are analyzed with the trained deep learning device, so as todetermine the tooth attribute value of said particular tooth model.

The allocation of the tooth attribute values may then be performedtotally without human intervention.

The step B) is intended for the acquisition of one or preferably severalupdated images.

The step B) preferably comprises one or more of the features of the stepb) of WO 2016 066651.

The acquisition of the updated images is performed by means of an imageacquisition apparatus, preferably chosen from a cellphone, a so-called“connected” camera, a so-called “smart” watch, a tablet or a personalcomputer, fixed or portable, comprising an image acquisition system suchas a webcam or a camera. Preferably, the image acquisition apparatus isa cellphone.

Also preferably, the image acquisition apparatus is separated from thedental arch by more than 5 cm, more than 8 cm, even more than 10 cm,which avoids the condensation of steam on the optic of the imageacquisition apparatus and facilitates the focusing. Furthermore,preferably, the image acquisition apparatus, in particular thecellphone, has no specific optic for the acquisition of the updatedimages, which is notably possible because of the separation of thedental arch during the acquisition.

Preferably, an updated image is a photograph or an image extracted froma film. It is preferably in color, preferably in real colors.

Preferably, the acquisition of the updated image or images is performedby the patient, preferably without the use of a support which is bearingon the ground and is immobilizing the image acquisition apparatus, andin particular without tripod.

In one embodiment, the triggering of the acquisition is automatic, thatis to say without the action of an operator, as soon as the acquisitionconditions are approved by the image acquisition apparatus, inparticular when the image acquisition apparatus has determined that itobserves a dental arch and/or a retractor and that the conditions ofobservation are satisfactory (sharpness, brightness, even dimensions ofthe representation of the dental arch and/or of the retractor).

The time interval between the steps A) and B) is as short as possible inorder for the teeth not to be substantially displaced between theproduction of the updated model and the acquisition of the updatedimages. Reference images matching with the updated images may then beacquired by observing the updated reference model.

Preferably, a dental retractor 10 is used in the step B), as representedin FIG. 12A. The retractor conventionally comprises a support providedwith a rim extending around an opening and arranged in such a way thatthe lips of the patient may rest thereon leaving the teeth of thepatient apparent through said opening.

In the step C), the updated reference model is explored to find, foreach updated image, a reference image exhibiting a maximal match withthe updated image.

The step C) may comprise one or more of the features of the steps c), d)and e) of WO 2016 066651, in as much as they concern such anexploration.

For each updated image, a set of virtual acquisition conditionsapproximating the real acquisition conditions upon the acquisition ofsaid updated image is preferably determined, roughly. In other words,the position of the image acquisition apparatus is estimated in relationto the teeth at the moment when it took the updated image (position ofthe acquisition apparatus in space and orientation of this apparatus).This rough assessment advantageously makes it possible to limit thenumber of tests on virtual acquisition conditions in subsequentoperations, and therefore makes it possible to considerably speed upthese operations.

To perform this rough assessment, one or more heuristic rules arepreferably used. For example, preferably, virtual acquisition conditionslikely to be tested in subsequent operations, the conditions whichcorrespond to a position of the image acquisition apparatus behind theteeth or at a distance from the teeth greater than 1 m, are excluded.

In a preferred embodiment, as illustrated in FIG. 13, registration marksare used that are represented on the updated image, and in particularregistration marks 12 of the retractor, to determine a region of thespace that is substantially conical delimiting virtual acquisitionconditions likely to be tested in subsequent operations, or “test cone”.

More specifically, at least three registration marks 12 are preferablypositioned non-aligned on the retractor 10, and their relative positionson the retractor are accurately measured.

The registration marks are then marked on the updated image, asdescribed previously. Simple trigonometric calculations make it possibleto determine approximately the direction from which the updated imagewas taken.

Next, for each updated image, a reference image is sought that exhibitsa maximum match with the updated image. This search is preferablyperformed by means of a metaheuristic method, preferably evolutionist,preferably by simulated annealing.

Preferably, at any instant before the step C4), the updated image isanalyzed so as to produce an updated map representing, at leastpartially, a discriminating information item. The updated map thereforerepresents the discriminating information item in the reference frame ofthe updated image.

The discriminating information item is preferably chosen from the groupconsisting of a contour information item, a color information item, adensity information item, a distance information item, a brightnessinformation item, a saturation information item, information onreflections and combinations of these information items.

A person skilled in the art knows how to process an updated image toreveal the discriminating information item.

For example, FIG. 12D is an updated map relating to the contour of theteeth obtained from the updated image of FIG. 12B.

Said search comprises the following steps:

-   -   C1) determination of virtual acquisition conditions “to be        tested”; C2) production of a reference image of the updated        reference model in said virtual acquisition conditions to be        tested;    -   C3) processing of the reference image to produce at least one        reference map representing, at least partially, the        discriminating information item;    -   C4) comparison of the updated and reference maps so as to        determine a value for an assessment function, said value for the        assessment function depending on the differences between said        updated and reference maps and corresponding to a decision to        continue or stop the search for virtual acquisition conditions        approximating said real acquisition conditions with greater        accuracy than said virtual acquisition conditions to be tested        determined in the last occurrence of the step C1);    -   C5) if said value for the assessment function corresponds to a        decision to continue said search, modification of the virtual        acquisition conditions to be tested, then return to the step        C2).

The step C1) begins with a determination of the virtual acquisitionconditions to be tested, that is to say a virtual position andorientation likely to correspond to the real position and orientation ofthe acquisition apparatus upon the capture of the updated image, butalso, preferably, a virtual calibration likely to correspond to the realcalibration of the acquisition apparatus upon the capture of the updatedimage.

The first virtual acquisition conditions to be tested are preferablyvirtual acquisition conditions assessed roughly, as describedpreviously.

In the step C2), the image acquisition apparatus is then virtuallyconfigured in the virtual acquisition conditions to be tested in orderto acquire a reference image of the updated reference model in thesevirtual acquisition conditions to be tested. The reference imagetherefore corresponds to the image that the image acquisition apparatuswould have taken if it had been placed, relative to the updatedreference model, and optionally calibrated, in the virtual acquisitionconditions to be tested.

If the updated image was taken substantially at the same moment as theupdated reference model was created by a scan of the teeth of thepatient, the position of the teeth on the updated image is substantiallyidentical to that in the updated reference model. If the virtualacquisition conditions to be tested are exactly the real acquisitionconditions, the reference image may therefore be exactly superimposed onthe updated image. The differences between the updated image and thereference image result from errors in the assessment of the virtualacquisition conditions to be tested, if they do not correspond exactlyto the real acquisition conditions.

In the step C3), the reference image is processed as the updated imageso as to produce, from the reference image, a reference map representingthe discriminating information item (FIGS. 11A and 11B). A personskilled in the art knows how to process a reference image to reveal thediscriminating information item.

In the step C4), the updated and reference maps, both relating to thesame discriminating information item, are compared and the difference or“distance” between these two maps is assessed by means of a score. Forexample, if the discriminating information item is the contour of theteeth, the mean distance between the points of the contour of the teethwhich appears on the reference image and the points of the correspondingcontour which appears on the updated image may be compared, the scorebeing all the higher when this distance is small.

The score may for example be a correlation coefficient.

Preferably, the virtual acquisition conditions comprise the calibrationparameters of the acquisition apparatus. The score is that much higherwhen the values of the calibration parameters tested are close to thecalibration parameter values of the acquisition apparatus used upon theacquisition of the updated image. For example, if the diaphragm aperturetested is far from that of the acquisition apparatus used upon theacquisition of the updated image, the reference image shows fuzzyregions and sharp regions which do not correspond to the fuzzy regionsand to the sharp regions of the updated image. If the discriminatinginformation item is the contour of the teeth, the updated and referencemaps will not therefore represent the same contours and the score willbe low.

The score is then assessed by means of an assessment function. Theassessment function makes it possible to decide whether the cycling onthe steps C1) to C5) must be continued or stopped. The assessmentfunction may for example be equal to 0 if the cycling must be stopped orbe equal to 1 if the cycling must continue.

The value of the assessment function may depend on the score achieved.For example, a decision may be made to continue the cycling if the scoredoes not exceed a threshold. For example, if an exact match between theupdated and reference images leads to a score of 100%, the threshold maybe, for example, 95%. Obviously, the higher the threshold, the betterwill be the accuracy of the assessment of the virtual acquisitionconditions if the score manages to exceed this threshold.

The value of the assessment function may also depend on scores obtainedwith virtual acquisition conditions tested previously.

The value of the assessment function may also depend on randomparameters and/or on the number of cycles already performed.

In particular, it is possible, despite the repetition of the cycles, notto manage to find virtual acquisition conditions which are sufficientlyclose to the real acquisition conditions for the score to reach saidthreshold. The assessment function may then lead to the decision to exitthe cycle even through the best score obtained has not reached saidthreshold. This decision may result, for example, from a number ofcycles greater than a predetermined maximum number.

A random parameter in the assessment function may also authorize thecontinuation of tests of new virtual acquisition conditions, even thoughthe score appears satisfactory.

The assessment functions conventionally used in the metaheuristicoptimization, preferably evolutionist, methods, particularly in thesimulated annealing methods, may be used for the assessment function.

In the step C5), if the value of the assessment function indicates adecision to continue the cycling, the virtual acquisition conditions tobe tested are modified and the cycling recommences on the steps C1) toC5), consisting in producing a reference image and a reference map, incomparing the reference map with the updated map to determine a score,then in taking a decision as a function of this score.

The modification of the virtual acquisition conditions to be testedcorresponds to a virtual displacement in space and/or to a modificationof the orientation and/or, preferably, to a modification of thecalibration of the acquisition apparatus. This modification may berandom, preferably such that the new virtual acquisition conditions tobe tested still belong to the set determined upon the rough assessment.The modification is preferably guided by heuristic rules, for example byfavoring the modifications which, according to an analysis of thepreceding scores obtained, appear most favorable for increasing thescore.

The cycling is continued until the value of the assessment functionindicates a decision to cease this cycling and to continue to the stepD), for example if the score reaches or exceeds said threshold.

The optimization of the virtual acquisition conditions is preferablyperformed by using a metaheuristic, preferably evolutionist, method,preferably a simulated annealing algorithm. Such an algorithm is wellknown for nonlinear optimization.

If the cycling has been exited without a satisfactory score having beenable to be obtained, for example without the score having been able toreach said threshold, the method may be stopped (failure situation) or anew step C) may be launched, with a new discriminating information itemand/or with a new updated image. The method may also be continued withthe virtual acquisition conditions corresponding to the best scorereached. A warning may be emitted in order to inform the user of theerror on the result.

If the cycling has been exited when a satisfactory score has been ableto be obtained, for example because the score has reached, even exceededsaid threshold, the virtual acquisition conditions correspondsubstantially to the real acquisition conditions of the updated image.

Preferably, the virtual acquisition conditions comprise the calibrationparameters of the acquisition apparatus. The method thus makes itpossible to assess the values of these parameters without it beingnecessary to know the nature of the acquisition apparatus or itssetting. The acquisition of the updated images may therefore beperformed without any particular precautions, for example by the patienthim or herself by means of his or her cellphone.

In addition, the search for the real calibration is performed bycomparing an updated image with views of a reference model in virtualacquisition conditions to be tested. Advantageously, it does not requirethe updated image to show a standard calibration gauge, that is to say agauge whose characteristics are accurately known making it possible todetermine the calibration of the acquisition apparatus.

The step C) therefore culminates in the determination of virtualacquisition conditions exhibiting a maximal match with the realacquisition conditions. The reference image therefore exhibits a maximalmatch with the updated image, that is to say that these two images maysubstantially be superimposed.

In one embodiment, said search of the virtual acquisition conditions inthe step C) is performed by using a metaheuristic, preferablyevolutionist, method, preferably a simulated annealing algorithm.

In the step D), the reference tooth zones are identified on thereference image and they are transferred to the updated image to definecorresponding updated tooth zones.

In particular, the reference image is a view of the updated referencemodel segmented into tooth models. The limits of the representation ofeach tooth model on the reference image, or “reference tooth zone”, maytherefore be identified.

The superimposition of the updated and reference images then makes itpossible to transfer the limits of the reference tooth zones to theupdated image, and thus define the updated tooth zones. Since thereference image exhibits a maximal match with the updated image, theupdated tooth zones therefore substantially define the limits of thetooth models represented on the reference image.

In the step E), each updated tooth zone is assigned the tooth attributevalue or values of the tooth model which corresponds to it.

In particular, the reference image is a view of the updated referencemodel in which the tooth models have been assigned respective toothattribute values for at least one tooth attribute, for example a toothnumber. Each reference tooth zone may therefore inherit the toothattribute value of the tooth model that it represents. Each updatedtooth zone may then inherit the tooth attribute value of the referencetooth zone which allowed it to be defined.

At the end of the step E), an updated image and a description of theupdated image is therefore obtained that defines one or more updatedtooth zones and, for each of these zones, a tooth attribute value for atleast one tooth attribute, for example a tooth number.

The updated image enriched with its description is called “historicalimage”.

FIG. 17A shows an example of updated image (acquired in the step B))being analyzed in order to determine the contours of the teeth. FIG. 17Bshows the reference image exhibiting a maximal match with the updatedimage (resulting from the step C)). The numbers of the teeth aredisplayed on the corresponding teeth. FIG. 17C illustrates the transferof the tooth numbers to the updated tooth zones (step D) and E)).

In the step F), the historical image is added to the learning base.

The steps A) to F) are preferably performed for more than 1000, morethan 5000, or more than 10 000 different patients, or “historicalpatients”.

As now clearly emerges, the invention provides a method that isparticularly effective for creating a learning base.

The invention relates also to a method for training a deep learningdevice, preferably a neural network, said method comprising anenrichment of a learning base according to a method comprising steps A)to F) so as to acquire a plurality of historical images, then the use ofsaid learning base to train said deep learning device.

Second Main Embodiment of the Enrichment Method

The invention is not however limited to the embodiments described above.

In particular, the updated reference model is not necessarily the directresult of a scan of the arch of the patient. The updated reference modelmay in particular be a model obtained by deformation of an initialreference model which is itself resulting directly from such a scan.

The method then comprises, preferably, in place of the steps A) to C),the steps A′) to C′).

The step A′) is identical to the step A). In the step A′), the referencemodel generated is however intended to be modified. It is thereforequalified as “initial reference model”, and not as “updated referencemodel”, as according to the step A).

The initial reference model may in particular be generated at an initialinstant preceding an active orthodontic treatment, for example less than6 months, less than 3 months, or less than 1 month before the start ofthe treatment. The steps B′) to C′) may then be implemented to track thetrend of the treatment between the initial instant and the updatedinstant of the step B′).

The initial instant may alternatively be an instant at the end of activeorthodontic treatment, for example less than 6 months, less than 3months, or less than 1 month after the end of the treatment. The stepsB′) to C′) may then be implemented to monitor the appearance of anyrecurrence.

The step B′) is identical to the step B). In the step B′), the updatedimages are however also intended to guide the modification of theinitial reference model to define the updated reference model, in thestep C′).

The time interval between the steps A′) and B′) is not limited, since,as explained hereinbelow, the initial reference model will be deformedto obtain an updated reference model with maximal match with the updatedimages. The time interval between the steps A′) and B′) may for examplebe greater than 1 week, than 2 weeks, than 1 month, than 2 months ormore than 6 months.

The step C′) is more complex than the step C) since the search for areference image exhibiting a maximal match with an updated image is notlimited to searching for the optimal virtual acquisition conditions. Italso includes a search for an updated reference model, that is to say areference model in which the teeth have substantially the same positionas on the updated image.

The step C′) preferably comprises one or more of the features of thesteps c), d) and e) of WO 2016 066651, and in particular of the step e)illustrated in FIG. 16.

The objective is to modify the initial reference model until an updatedreference model is obtained which exhibits a maximal match with theupdated image. Ideally, the updated reference model is therefore a modelof the arch from which the updated image would have been able to betaken if this model had been the arch itself.

A succession of reference models “to be tested” is therefore tested, thechoice of a reference model to be tested preferably being dependent onthe level of correspondence of the reference models “to be tested”previously tested with the updated image.

Preferably, the search comprises, for an updated image,

-   -   a first optimization operation making it possible to search, in        a reference model to be tested determined from the initial        reference model, for virtual acquisition conditions best        corresponding to the real acquisition conditions of the updated        image, and    -   a second optimization operation making it possible to search, by        testing a plurality of said reference models to be tested, for        the reference model best corresponding to the positioning of the        teeth of the patient upon the acquisition of the updated image.

Preferably, a first optimization operation is performed for each test ofa reference model to be tested during the second optimization operation.

Preferably, the first optimization operation and/or the secondoptimization operation, preferably the first optimization operation andthe second optimization operation, implement a metaheuristic method,preferably evolutionist, preferably a simulated annealing.

The step C′) therefore culminates in the determination

-   -   of an updated reference model exhibiting a maximal match with        the updated image, and    -   virtual acquisition conditions exhibiting a maximal match with        the real acquisition conditions.

A method comprising the steps A′) to C′) may advantageously beimplemented in the context of an active or passive orthodontictreatment, or, more generally, to track any changes to the teeth.

In the different methods according to the invention, the enrichment ofthe learning base does not necessarily result from an enrichment methodaccording to the invention.

In one embodiment, the learning base is created by an operator. Thelatter thus analyzes thousands of analysis images. For the learning baseto be able to be used for the implementation of a detailed analysismethod, it determines the tooth zones, then assigns them tooth attributevalues. For the learning base to be able to be used for theimplementation of a global analysis method, it assigns image attributevalues to each image. It may thus form historical images.

Detailed Image Analysis Method

The method for detailed analysis of an “analysis image” of a dental archof a patient according to the invention comprises the steps 1) to 4).

Preferably, the analysis image, preferably a photograph or an imageextracted from a film, preferably in color, preferably in real color, isacquired with an image acquisition apparatus, preferably a cellphone,separated from the dental arch by more than 5 cm, more than 8 cm, evenmore than 10 cm, and which, preferably, has no other specific optic.

Preferably, the analysis image represents several teeth, preferably morethan 2, more than 3, more than 4 or more than 5 teeth of the patient.FIG. 12A or FIG. 12B could be examples of analysis images. Thearrangement of the teeth is realistic, i.e. it corresponds to thearrangement which is observed by the image acquisition apparatus when ithas acquired the analysis image.

Preferably, the acquisition of the analysis image is performed by thepatient, preferably without the use of a support which is bearing on theground and is immobilizing the image acquisition apparatus, and inparticular without tripod.

In one embodiment, the triggering of the acquisition of the analysisimage is automatic, that is to say without the action of an operator, assoon as the acquisition conditions are approved by the image acquisitionapparatus, in particular when the image acquisition apparatus hasdetermined that it observes a dental arch and/or a retractor and thatthe observation conditions are satisfactory (sharpness, brightness, evendimensions of the representation of the dental arch and/or of theretractor).

In the step 1), a learning base is created comprising more than 1000,preferably more than 5000, preferably more than 10 000, preferably morethan 30 000, preferably more than 50 000, preferably more than 100 000historical images. The greater the number of historical images, thebetter the analysis performed by the method.

Preferably, a learning base is used that is enriched according to anenrichment method according to the invention.

The learning base may however be constructed according to other methods,for example be created manually. To create a historical image of thelearning base, an operator, preferably an orthodontist, identifies oneor more “historical” tooth zones on an image, called then assigns, toeach identified historical tooth zone, a value for at least one toothattribute.

In step 2), a deep learning device, preferably a neural network, istrained with the learning base.

A “neural network” or “artificial neural network” is a set of algorithmswell known to a person skilled in the art.

The neural network may in particular be chosen from:

-   -   the networks specializing in the classification of images,        called “CNN” (“convolutional neural network”), for example        -   AlexNet (2012)        -   ZF Net (2013)        -   VGG Net (2014)        -   GoogleNet (2015)        -   Microsoft ResNet (2015)        -   Caffe: BAIR Reference CaffeNet, BAIR AlexNet        -   Torch:VGG_CNN_S,VGG_CNN_M,VGG_CNN_M_2048,VGG_CNN_M_10            24,VGG_CNN_M_128,VGG_CNN_F,VGG ILSVRC-2014 16-layer,VGG            ILSVRC-2014 19-layer,Network-in-Network (Imagenet &            CIFAR-10)        -   Google: Inception (V3, V4).    -   The networks specializing in the location and detection of        objects in an image, the object detection network, for example:        -   R-CNN (2013)        -   SSD (Single Shot MultiBox Detector: Object Detection            network), Faster R-CNN (Faster Region-based Convolutional            Network method: Object Detection network)        -   Faster R-CNN (2015)        -   SSD (2015).

The above list is not limiting

In the step 2), the deep learning device is preferably trained by alearning process called “deep learning”. By presenting, as input for thedeep learning device, historical images (images+descriptions), the deeplearning device progressively learns to recognize patterns on an image,and to associate them with tooth zones and with tooth attribute values,for example tooth numbers.

In the step 3), the image that is to be analyzed, or “analysis image”,is submitted to the deep learning device.

Through its training in the step 2), the deep learning device is capableof analyzing the analysis image and of recognizing said patternstherein. It may in particular determine a probability relating to:

-   -   the presence, at a location in said analysis image, of a zone        representing, at least partially, a tooth, or “analysis tooth        zone”,    -   the attribute value of the tooth represented on said analysis        tooth zone.

For example, it is capable of determining that there is a 95% chancethat a form of the analysis image represents an incisor.

Preferably, the deep learning device analyses all the analysis image anddetermines probabilities for all the analysis tooth zones that it hasidentified.

In the step 4), the results of the preceding step are analyzed todetermine the teeth represented on the analysis image.

When the learning base comprises more than 10 000 historical images, thestep 3) gives particularly satisfactory results. In particular, such alearning base makes it possible to establish a probability thresholdsuch that, if a probability associated with an analysis tooth zone andwith a tooth attribute value for this analysis tooth zone exceeds saidthreshold, the analysis tooth zone effectively represents a tooth havingsaid tooth attribute value.

The step 4) thus leads to the definition of an analysis image enrichedwith a description defining the analysis tooth zones and, for eachanalysis tooth zone, the values of the attributes of the toothrepresented by the analysis tooth zone.

Global Image Analysis Method

The method for global analysis of an updated image of a dental arch of apatient according to the invention comprises the steps 1′) to 3′).

The method is similar to the detailed analysis method described above,apart from the fact that, according to the global analysis, it is notnecessary to analyze the individual situation of each tooth. Theanalysis is global to all the image. In other words, the deep learningdevice determines the value of an “image” attribute without having topreviously determine tooth attribute values.

For example, the deep learning device may conclude that, “globally”, thedental situation is “satisfactory” or “unsatisfactory”, withoutdetermining the tooth potentially at the origin of the dissatisfaction.

The step 1′) is similar to the step 1). The historical images howeverinclude a description specifying an image attribute value for eachimage.

The step 2′) is similar to the step 2).

In the step 3′), the analysis image is submitted to the deep learningdevice.

Through its training in the step 2′), the deep learning device iscapable of analyzing the analysis image and of recognizing said patternstherein. Based on these patterns, it may in particular determine aprobability relating to the value of the image attribute considered.

Application to the Modeling of a Dental Arch

A detailed analysis method according to the invention is in particularuseful for modeling a dental arch, notably for establishing a remotediagnostic.

It is desirable for each to regularly check his or her dentition,particularly in order to check that the position of his or her teeth isnot changing for the worse. In an orthodontic treatment, this change forthe worse may in particular lead to the treatment being modified. Afteran orthodontic treatment, a change for the worse, called “recurrence”,may lead to a repeat of a treatment. Finally, more generally andindependently of any treatment, each may want to track any displacementsof his or her teeth.

Conventionally, the checks are performed by an orthodontist who hasappropriate apparatus. These checks are therefore costly. Furthermore,the appointments are restrictive. Finally, some people are apprehensiveof a visit to an orthodontist and will fail to make an appointment for asimple check or to assess the feasibility of an orthodontic treatment.

US 2009/0291417 describes a method making it possible to create, thenmodify, three-dimensional models, particularly for the manufacture oforthodontic appliances.

WO 2016 066651 describes a method for checking the positioning and/orthe shape and/or the appearance of teeth of a patient. This methodcomprises a step of creation of an initial reference model of the teeth,at an initial instant, preferably with a 3D scanner, then, at asubsequent instant, or “updated instant”, for example six months afterthe initial instant, the creation of an updated reference model, bydeformation of the initial reference model. This deformation isperformed in such a way that the updated reference model allowsobservation substantially identical to images of the teeth acquired atthe updated instant, in particular to photos or images from a videotaken by the patient him or herself, with no particular precautions,called “updated images”.

The updated images are therefore used to modify the initial, veryaccurate, reference model. The updated reference model which resultsfrom the deformation of the initial reference model, guided by theanalysis of the updated images, is therefore also very accurate.

The method described in WO 2016/66651 does however require anappointment with the orthodontist in order to create the initialreference model. This appointment constitutes a brake to prevention. Ineffect, a patient will not necessarily consult an orthodontist if he orshe does not perceive the need therefor. In other words, the method isoften implemented only if a malocclusion is found and has to becorrected.

There is therefore a need for a method that addresses this problem, byfacilitating prevention.

One aim of the invention is to address this need.

To this end, the invention proposes a method for modeling a dental archof a patient, said method comprising the following steps:

-   -   a) creation of a historical library comprising more than 1000        tooth models, called “historical tooth models”, and assignment        to each historical tooth model of a value for at least one tooth        attribute, or “tooth attribute value”;    -   b) analysis of at least one “analysis image” of the dental arch        according to a detailed analysis method according to the        invention, so as to determine at least one analysis tooth zone        and at least one tooth attribute value associated with said        analysis tooth zone;    -   c) for each analysis tooth zone determined in the preceding        step, search, in the historical library, for a historical tooth        model exhibiting a maximum proximity with the analysis image or        with the analysis tooth zone, or “optimal tooth model”;    -   d) arrangement of all the optimal tooth models so as to create a        model which exhibits a maximal match with the updated image, or        “assembled model”;    -   e) optionally, replacement of at least one optimal tooth model        by another historical tooth model and repeat of the step d) so        as to maximize the match between the assembled model and the        analysis image;    -   f) optionally, repeat of the step b) with another analysis image        and, in the step d) and/or e), search for a maximal match with        all the analysis images used.

The invention thus makes it possible, from a simple analysis image, forexample a photograph taken by means of a cellphone, to reconstruct, witha good reliability, a dental arch in the form of an assembled model. Inparticular, the analysis image may be acquired as described here aboveat step 1).

Obviously, the analysis of a single analysis image is insufficient togenerate an assembled model which accurately corresponds to thearrangement of the teeth of the patient. Such an accuracy is however notgenerally essential for performing a first diagnosis of the dentalsituation of the patient.

Moreover, the accuracy of the assembled model may be increased ifseveral analysis images are processed.

The steps b) to c) are preferably implemented for several analysisimages and, in the steps d) and e), the optimal tooth models and anassembled model are searched so as to obtain a maximal match in light ofall the analysis images (step f)).

The invention also relates to a method for assessing a dental situationof a patient, comprising the following steps:

-   -   i) creation of an assembled model according to a modeling method        according to the invention;    -   ii) transmission of the assembled model to a recipient,        preferably an orthodontist and/or a computer;    -   iii) analysis of the orthodontic situation of the patient, by        the recipient, from the assembled model;    -   iv) preferably, informing of the patient of the orthodontic        situation, preferably, via his or her cellphone.

The patient may therefore very easily ask an orthodontist to check hisor her dental situation, without even having to travel, by simplytransmitting one or preferably several photos of his or her teeth.

A modeling method is now described in detail.

In the step a), a historical library 20 (FIG. 18) is created comprisingmore than 1000, preferably more than 5000, preferably more than 10 000historical tooth models 22. The greater the number of historical toothmodels, the more accurate the assembled model.

A historical tooth model can in particular be obtained from a model of adental arch of a “historical” patient obtained with a scanner. This archmodel may be segmented in order to isolate the representations of theteeth, as in FIG. 11D. Each of these representations, exhibiting aspecific gray shade in FIG. 11D, can constitute a historical toothmodel.

Preferably, the library is enriched with the tooth models resulting fromthe implementation of the method described in WO 2016 066651 or of thestep A) or A′) described above.

One or several tooth attributes, in particular chosen from the listsupplied above, are associated with the tooth models. A tooth attributevalue is assigned to each tooth attribute of a particular tooth model,as described previously (see the description of the step A)). Forexample, a tooth model is that of an “incisor”, “greatly worn”, andwhose color parameters are, in the color system L*a*b* according to thestandard NF ISO 7724, “a*=2”, “b*=1” and “L*=58”.

The historical library therefore contains historical tooth models andassociated attribute values which facilitate the search in the step c).In FIG. 18, only historical tooth models 22 representing molars havebeen represented in the historical library 20.

In the step b), an analysis image is acquired, as described above forthe step B), before analyzing it. In particular, the analysis image ispreferably a photo or an image from a film, preferably with a cellphone.

The analysis image can be acquired at any moment after the step a), forexample more than one week, more than one month or more than six monthsafter the step a).

The analysis image is analyzed according to a detailed analysis methodaccording to the invention. The optional features of this method arealso optional in the step b).

On completion of the step b), an analysis image is obtained that isenriched with a description providing, for each analysis tooth zone, atooth attribute value for at least one tooth attribute, for example atooth number.

In the step c), a search is carried out in the historical library, foreach analysis tooth zone determined in the preceding step, for ahistorical tooth model exhibiting a maximal proximity with the analysistooth zone. This tooth model is qualified as “optimal tooth model”.

The “proximity” is a measurement of one or more differences between thehistorical tooth model and the analysis tooth zone. These differencesmay include a difference in shape, but also other differences like adifference in translucency or in color. The maximal proximity may besearched for by successively minimizing several differences, or byminimizing a combination of these differences, for example a weightedsum of these differences.

The “proximity” is therefore a broader concept than “match”, the matchmeasuring only a shape-related proximity

The assessment of the proximity of a historical tooth model with ananalysis tooth zone preferably comprises a comparison of at least onevalue of a tooth attribute of the analysis tooth zone with the value ofthis attribute for the historical tooth model. Such an assessment isadvantageously very rapid.

For example, if the description of the analysis tooth zone provides avalue for the type or the number of the tooth, the thickness of thetooth represented and/or the height of its crown and/or its mesiopalatine width and/or the mesial and distal deflection index of itsincisal edge, this value may be compared to the value of thecorresponding attribute of each of the historical tooth models.

Preferably, a historical tooth model is sought that has, for at leastone tooth attribute, the same value as said analysis tooth zone. Thetooth attribute may in particular relate to the tooth type or to thetooth number. In other words, the historical tooth models are filteredto examine in more detail only those which relate to the same type oftooth as the tooth represented on the analysis tooth zone.

Alternatively or, preferably, in addition to this comparison of theattribute values, the shape of the tooth represented on the analysistooth zone may be compared to the shape of a historical tooth model tobe assessed, preferably by means of a metaheuristic method, preferablyevolutionist, preferably by simulated annealing.

To this end, the historical tooth model to be assessed is observed fromdifferent angles. Each view thus obtained is compared with the analysisimage, preferably with the analysis tooth zone, so as to establish a“distance” between this view and said analysis image or, preferably,said analysis tooth zone. The distance thus measures the differencebetween the view and the analysis tooth zone.

The distance may be determined after a processing of the view and of theanalysis image or, preferably, of the analysis tooth zone, so as toreveal, on corresponding maps, one and the same discriminatinginformation item, for example a contour information item, as describedabove in the step C3) or in WO 2016 066651.

For each historical tooth model tested, a view is thus determined whichprovides a minimal distance with the analysis image or with the analysistooth zone. Each historical tooth model examined is thus associated witha particular minimal distance, which measures its proximity of shapewith the analysis tooth zone.

The optimal historical tooth model is the one which, in light of thecomparison or comparisons performed, is considered to be closest to theanalysis tooth zone.

The minimal distances obtained for the different tooth models tested arethen compared and, to define the optimal tooth model, the one whichexhibits the smallest minimal distance is retained. The optimal toothmodel therefore exhibits a maximal match with the analysis image.

The search for the maximal match is preferably performed by means of ametaheuristic method, preferably evolutionist, preferably by simulatedannealing.

In a preferred embodiment, a first assessment of the historical toothmodels by comparison of the values of at least one tooth attribute, forexample the tooth number, with the corresponding values of the analysistooth zone, then a second assessment by comparison of shape areperformed in succession. The first, rapid assessment, advantageouslymakes it possible to filter the historical tooth models in order tosubmit only the historical tooth models retained by the first assessmentto the second, slower assessment.

For example, if an analysis tooth zone represents a tooth number 15, thefirst assessment makes it possible to retain only the tooth modelsmodeling teeth number 15. During the second assessment, a search isconducted, among all the historical tooth models modeling teeth number15, for the historical tooth model whose shape most approximates that ofthe tooth represented.

More preferably, several first assessments are performed beforeperforming the second assessment. For example, the first assessmentsmake it possible to filter the historical tooth models so as to retainonly the tooth models modeling teeth number 15 and which exhibit a crownheight of between 8 and 8.5 mm.

On completion of the step c), an optimal tooth model has thus beenassociated with each of the analysis tooth zones.

For example, in FIG. 18, the historical tooth model 22 ₁ can be observedso as to strongly resemble an analysis zone identified on the analysisimage. It is considered as optimal for this analysis zone.

In the step d), an assembled model is created by arranging the optimaltooth models.

According to one embodiment, at the start of the step d), a first rougharrangement is created, that is to say that a rough model is produced byassembling optimal tooth models.

To establish the first rough arrangement, the optimal tooth models canbe oriented so that their optimal directions of observation are allparallel, the optimal direction of observation of a tooth model beingthe direction in which said tooth model exhibits a maximum match withthe analysis image.

The first rough arrangement may be also established by considering thetooth attribute values of the optimal tooth models. For example, if thetooth numbers of optimal tooth models are those of the canines and ofthe incisors, these tooth models may be arranged according to an arc 24(FIG. 18) corresponding conventionally to the region of the arch whichbears these types of teeth.

The shape of this arc may be refined as a function of other toothattribute values.

The order of the optimal tooth models is that of the correspondinganalysis tooth zones.

Moreover, the minimal distance associated with an optimal tooth modelresults from an observation of the tooth model according to an “optimal”observation direction. In other words, it is probably substantially fromthis direction that the tooth that this model models is also observed inthe analysis image. All the optimal tooth models are thus preferablyoriented in such a way that their respective optimal observationdirections are all parallel.

It is thus possible to define a first arrangement of the optimal toothmodels.

Preferably, the first arrangement of the optimal tooth models is thenmodified iteratively, so as to exhibit a maximal match with the analysisimage.

To assess an arrangement, it is observed according to different angles.Each view thus obtained is compared with the analysis image so as toestablish a “distance” between this view and said analysis image. Thedistance thus measures the difference between the view and the analysisimage.

The distance may be determined after a processing of the view and of theanalysis image so as to reveal, on one of the corresponding maps, adiscriminating information item, for example a contour information item,as described above in the step C3) or in WO 2016 066651.

For each arrangement examined, a view is thus determined that provides aminimal distance with the analysis image. Each arrangement examined isthus associated with a minimal distance.

The minimal distances obtained for the different arrangements tested arethen compared and, to define the optimal arrangement, the one whichexhibits the smallest minimal distance is retained. The optimalarrangement thus exhibits a maximal match with the analysis image.

The search for the maximal match is performed preferably by means of ametaheuristic method, preferably evolutionist, preferably by simulatedannealing.

On completion of the step d), an optimal arrangement of the optimaltooth models, that is to say the assembled model 26, is obtained.

In the optional step e), one or more optimal tooth models are replacedby other tooth models, then there is a return to the step d) so as tomaximize the match between the assembled model and the analysis image.

It is in fact possible for an optimal tooth model, in the “optimal”arrangement, to no longer exhibit a maximal match with the analysisimage. In particular, the tooth model might have been observed in an“optimal” direction which provided a view exhibiting a minimal distancewith the analysis image (reason why it was considered as optimal), but,in the optimal arrangement, it is no longer oriented according to theoptimal direction.

A new search for an assembled model may therefore be performed bymodifying the tooth models, for example by replacing the optimal toothmodels with close tooth models.

The search for the tooth models to be tested is preferably performed bymeans of a metaheuristic method, preferably evolutionist, preferably bysimulated annealing.

In the preferred embodiment, the method therefore implements a doubleoptimization, on the tooth models and on the arrangement of the toothmodels, the assembled model being the arrangement of a set of toothmodels which provides the minimal distance with the analysis image, byconsidering all the possible tooth models and all the possiblearrangements.

In the optional and preferred step f), the method implements severalanalysis images of the arch of the patient, preferably more than 3, morethan 5, more than 10, more than 50, preferably more than 100 analysisimages. The assembled model is thus more complete. More preferably, themethod implements an optimization such that the assembled model obtainedis optimal in light of all of the analysis images. In other words, theassembled model is preferably the one which maximizes the match with allof the analysis images.

In the preferred embodiment, the method therefore implements a doubleor, preferably a triple optimization, on the tooth models on the onehand, on the arrangement of the tooth models and/or on a plurality ofanalysis images on the other hand, the assembled model being thearrangement of a set of tooth models which provides the mean minimaldistance, over all of the analysis images, by considering all thepossible tooth models and, preferably, all the possible arrangements.

According to one embodiment, in the step d) and/or e) and/or f), ametaheuristic, preferably evolutionist, method, preferably by simulatedannealing, is used.

As now clearly appears, the invention thus makes it possible toconstruct an assembled dental arch model from simple analysis images,for example photographs taken by means of a cellphone. Obviously, theaccuracy of the assembled model does not reach that of a scan. In someapplications, for example to perform a first diagnosis of the dentalsituation of the patient, such an accuracy is not however essential.

The assembled model may therefore be used to analyze the orthodonticsituation of the patient, according to the steps ii) to iv).

In the step ii), the assembled model is sent to an orthodontist and/orto a computer provided with diagnostic software.

In one embodiment, the assembled model is sent accompanied by aquestionnaire filled out by the patient in order to improve the qualityof the analysis in the step iv).

In the step iii), the orthodontist and/or the computer examines theassembled model. Unlike an updated image, the assembled model allows anobservation from any angle. The analysis is advantageously moreaccurate.

In the step iv), the orthodontist and/or the computer informs thepatient, for example by sending him or her a message on his or hertelephone. This message may in particular inform the patient of anunfavorable situation and urge him or her to make an appointment withthe orthodontist.

The orthodontist may also compare the assembled model with assembledmodels received previously for the same patient. The analysis thereofadvantageously makes it possible to assess the trend of the situation.The message may thus inform the patient of an unfavorable change of hisor her situation, which improves the prevention.

The assembled model may also be compared with one or more modelsobtained by scanning of the teeth or from a molding of the teeth of thepatient, or with an updated reference model resulting from theimplementation of a method described in WO 2016 066651.

Application to Embedded Monitoring

An image analysis according to the invention is also useful for guidingthe acquisition of an image of a dental arch, in particular forestablishing a remote diagnosis.

In particular, WO 2016/066651 describes a method in which an initialreference model is deformed so as to obtain an updated reference modelallowing the acquisition of reference images exhibiting a maximal matchwith the “updated” images of the arch acquired at the updated instant.

The reference images are therefore views of the updated reference model,observed in virtual acquisition conditions which are the closestpossible matches with the real acquisition conditions implemented toacquire the updated images of the arch of the patient.

The search for these virtual acquisition conditions is preferablyperformed by means of metaheuristic methods.

To speed up this search, WO 2016/066651 recommends performing a firstrough assessment of the real acquisition conditions. For example,conditions which would correspond to a position of the acquisitionapparatus at a distance from the teeth greater than 1 meter are excludedfrom the search.

There is however an ongoing need to speed up the execution of the methoddescribed in WO 2016/066651, and in particular, to search, more rapidly,for the virtual acquisition conditions exhibiting a maximal match withthe real acquisition conditions implemented to acquire an updated imageof the arch of the patient.

One aim of the invention is to at least partially address this problem.

The invention proposes a method for acquiring an image of a dental archof a patient, said method comprising the following steps:)

-   -   a′) activation of an image acquisition apparatus so as to        acquire an image, called “analysis image”, of said arch;    -   b′) analysis of the analysis image by means of a deep learning        device, preferably a neural network, trained by means of a        learning base,        -   preferably according to a detailed analysis method according            to the invention, so as to identify at least one analysis            tooth zone representing a tooth on said analysis image, and            to determine at least one tooth attribute value for said            analysis tooth zone, or according to a global analysis            method according to the invention;    -   c′) determination, for the analysis image, of a value for an        image attribute, said value being a function of said tooth        attribute value if a detailed analysis method according to the        invention has been implemented in the preceding step;    -   d′) optionally, comparison of said image attribute value with an        instruction;    -   e′) sending of an information message as a function of said        comparison.

In one embodiment, in the step b′), all said analysis tooth zones areidentified, and at least one tooth attribute value is determined foreach analysis tooth zone, and, in the step c′), the value for the imageattribute is determined as a function of said tooth attribute values.

In one embodiment, the step b′) comprises the following steps:

-   -   1) preferably before the step a′), creation of a learning base        comprising more than 1 000 dental arch images, or “historical        images”, each historical image comprising one or more zones each        representing a tooth, or “historical tooth zones”, to each of        which, for said tooth attribute, a tooth attribute value is        assigned;    -   2) training of at least one deep learning device, preferably a        neural network, by means of the learning base;    -   3) submission of the analysis image to the deep learning device        so that it determines at least one probability relating to:        -   the presence, at a location in said analysis image, of a            zone representing, at least partially, a tooth, or “analysis            tooth zone”,        -   the attribute value of the tooth represented on said            analysis tooth zone,    -   4) determination, as a function of said probability, of the        presence of a tooth at a position represented by said analysis        tooth zone, and of the attribute value of said tooth.

In one embodiment, the step b′) comprises the following steps:

1′) creation of a learning base comprising more than 1 000 dental archimages, or “historical images”, each historical image comprising anattribute value for at least one image attribute, or “image attributevalue”;

-   -   2′) training of at least one deep learning device, preferably a        neural network, by means of the learning base;    -   3′) submission of the analysis image to the deep learning device        so that it determines, for said analysis image, at least one        probability relating to said image attribute value.

In one embodiment, to create a historical image of the learning base, anoperator, preferably an orthodontist,

-   -   identifies one more “historical” tooth zones on an image, then        assigns each identified historical tooth zone a value for at        least one tooth attribute, and/or    -   assigns an image a value for at least one tooth attribute.

In one embodiment, the information message is sent by the acquisitionapparatus.

As will be seen in more detail hereinafter in the description, anacquisition method according to the invention therefore makes itpossible to check whether an analysis image respects an instruction and,if it does not respect the instruction, to guide the operator in orderfor him or her to acquire a new analysis image. The method thereforeallows an “embedded monitoring”, preferably in the image acquisitionapparatus.

In particular, to implement the method of WO 2016/066651, there may be adesire to acquire updated images from different acquisition directions,for example a front image, a right image and a left image. These updatedimages, acquired in succession, may be classified accordingly. Thesearch for the virtual acquisition conditions exhibiting a maximal matchwith the real acquisition conditions is speeded up thereby.

In effect, the search may begin from virtual acquisition conditions inwhich the virtual acquisition apparatus is in front, to the left or tothe right of the updated reference model, depending on whether theupdated image considered is classified as a front, left or right image,respectively.

The operator, generally the patient, may however make a mistake in theacquisition of the updated images. In particular, he or she may forgetto take an updated image, for example the front view, or invert twoupdated images. Typically, the operator may take an image on the rightwhereas a left image is expected from him or her.

This inversion of the updated images may considerably slow down theirprocessing. For example, if the updated image is assumed to be an imagetaken on the left but it was in error taken on the right, said searchfor the optimal virtual acquisition conditions, that is to say thoseexhibiting a maximal match with the real acquisition conditions, willbegin from a starting point offering a left view of the reference model,whereas the optimal virtual acquisition conditions correspond to a rightview. The search will therefore be considerably slowed down thereby.

By virtue of the invention, each updated image is an analysis imagewhich may be analyzed and checked, preferably in real time.

For example, the acquisition method makes it possible to determine thatthe updated image has been “taken on the right” and compare this imageattribute value with the instruction which had been given to theoperator to take the updated image on the left. Since the attributevalue of the updated image (image taken on the right) does notcorrespond to the instruction (acquire an updated image on the left),the acquisition apparatus may immediately alert the operator in orderfor him or her to modify the acquisition direction.

An acquisition method is now described in detail.

In the step a′), the operator activates the image acquisition apparatusso as to acquire an analysis image.

In one embodiment, the operator triggers the acquisition apparatus so asto store the analysis image, preferably takes a photo or video of his orher teeth, preferably by means of a cellphone equipped with a camera.

The step a′) may be performed like the acquisition of the updated imagesin the step B) described above.

In another embodiment, the analysis image is not stored. In particular,the analysis image may be the image which, in real time, appears on thescreen of the cellphone of the operator, generally the patient.

In a first embodiment, in the step b′), the analysis image is analyzedaccording to a detailed analysis method according to the invention. Thisanalysis preferably leads to the assignment of a tooth attribute valueto each identified analysis tooth zone, for example to assigning a toothnumber to each of the analysis tooth zones.

In the step c′), an attribute value of the analysis image is determinedas a function of the tooth attribute values. The attribute value of theanalysis image may relate to its general orientation and may for exampletake one of the following three values: “right photo”, “left photo” and“front photo”. The attribute value of the analysis image may also be thelist of the tooth numbers represented, for example, “16, 17 and 18”. Theattribute value of the analysis image may also be, for example, the“presence” or “absence” of a dental, preferably orthodontic, appliance,or the state of opening of the mouth (“mouth open”, “mouth closed”).

In another embodiment, a global analysis method according to theinvention is implemented in the step b′). Advantageously, such a methodmakes it possible to directly obtain a value for an image attribute,without having to determine values for a tooth attribute. It istherefore advantageously more rapid. The information resulting from aglobal analysis may however be less accurate than that resulting from adetailed analysis.

The steps a′) to c′) thus make it possible to characterize the analysisimage.

The characterization of the analysis image makes it possible to guidethe operator if the analysis image does not correspond to the expectedimage, for example because its quality is insufficient or because itdoes not represent the desired teeth.

In the step d′), the image attribute value of the analysis image iscompared with an instruction.

For example, if the instruction was to acquire a right image and theimage attribute value is “take on the left”, the comparison leads to theconclusion that the image acquired is “unsatisfactory”.

In the step e′), a message is sent to the operator, preferably by theacquisition apparatus.

Preferably, the information message relates to the quality of the imageacquired and/or to the position of the acquisition apparatus in relationto said arch and/or to the setting of the acquisition apparatus and/orto the opening of the mouth and/or to the wearing of a dental,preferably orthodontic, appliance.

For example, if the image acquired is “unsatisfactory”, the acquisitionapparatus may emit a light, for example red, and/or ring, and/orgenerate a voice message, and/or vibrate, and/or display a message onits screen.

For example, if the image has to be acquired while the patient wears hisor her dental appliance and this is not the case, the acquisitionapparatus may send the message “wear your appliance for this image”.

For example, if the image was acquired with the patient not sufficientlyopening the mouth or with the mouth closed, the acquisition apparatusmay send the message “open your mouth more for this image”.

In one embodiment, the steps b′) to c′) are implemented only if theoperator records the analysis image, that is to say he or she presses onthe trigger. The message then prompts the operator to acquire a newanalysis image. Optionally, the acquisition apparatus deletes theunsatisfactory analysis image.

In one embodiment, the steps b′) to c′) are implemented permanently whenthe acquisition apparatus is switched on and the analysis image is animage which appears on a screen of the acquisition apparatus. Theacquisition apparatus may thus, for example, emit a red light as long asthe analysis image is unsatisfactory, and emit a green light when theanalysis image is satisfactory. Advantageously, the acquisitionapparatus then stores only analysis images which are satisfactory.

As now clearly emerges, the invention therefore allows an embeddedmonitoring upon the acquisition of analysis images. When applied toupdated images of the method of WO 2016/066651, the steps a′) to e′)make it possible to check that these images do indeed conform to theneed, and therefore to considerably speed up the execution of thismethod.

The steps d′) and e′) are optional. In one embodiment, the analysisimage is only associated with its description, which specifies its imageattribute value. This description also makes it possible to considerablyspeed up the execution of the method of WO 2016/066651 since, when theanalysis image is used as updated image of this method, it makes itpossible to approximately determine the real acquisition conditions ofthis image, by eliminating the risk of a rough error, for examplebecause of a reversal between two images.

The steps d′) and e′) are however preferred. They make it possible forexample to avoid having the operator forget a left image, or take tworedundant right images.

Application to the Monitoring of an Orthodontic Aligner

Conventionally, at the start of an orthodontic treatment, theorthodontist determines the positioning of the teeth that he or shewants to obtain at a so-called “set-up” instant of the treatment. Theset-up may be defined by means of an imprint or from a three-dimensionalscan of the teeth of the patient. The orthodontist then hasmanufactured, or manufactures, accordingly, an orthodontic appliancesuited to this treatment.

The orthodontic appliance may be an orthodontic aligner. An alignerconventionally takes the form of a removable single-piece appliance,conventionally made of a transparent polymer material, which comprises achannel conformed for several teeth of an arch, generally all the teethof an arch, to be able to be housed therein.

The form of the channel is adapted to hold the aligner in position onthe teeth, while exerting an action of correction of the positioning ofcertain teeth (FIGS. 14 and 15).

At the start of the treatment, the shapes that the different alignershave to take at different moments of the treatment are conventionallydetermined, then the set of corresponding aligners is manufactured. Atpredetermined instants, the patient changes the aligner.

The treatment by means of aligners is advantageously less stressful forthe patient. In particular, the number of appointments with theorthodontist is limited. Furthermore, there is less pain than with ametal orthodontic brace attached to the teeth.

The market for orthodontic aligners is therefore on the increase.

At regular intervals, the patient goes to the orthodontist for a visualinspection, notably to check whether the displacement of the teethconforms to the expectations and whether the aligner worn by the patientis still suited to the treatment.

If the orthodontist diagnoses an unsuitability to the treatment, he orshe takes a new imprint of the teeth, or, in an equivalent manner, a newthree-dimensional scan of the teeth, then orders a new series ofaligners configured accordingly. It is considered that, on average, thenumber of aligners finally manufactured is approximately 45, instead of20 aligners conventionally provided at the start of the treatment.

The need to have to go to the orthodontist is a constraint for thepatient. The trust of the patient in his or her orthodontist may also beaffected. The inadequacy may be unsightly. Finally, the result thereofis an additional cost.

The number of inspection visits to the orthodontist must therefore belimited.

There is a need for solutions addressing these problems.

One aim of the invention is to at least partially address this need.

The invention provides a method for assessing the shape of anorthodontic aligner, said method comprising the following steps:

-   -   a″) acquisition of at least one image at least partially        representing the aligner in a service position in which it is        worn by a patient, called “analysis image”;    -   b″) analysis of the analysis image by means of a deep learning        device, preferably a neural network, trained by means of a        learning base, so as to determine a value for at least one tooth        attribute of an “analysis tooth zone” of the analysis image, the        tooth attribute relating to a separation between the tooth        represented by the analysis tooth zone, and the aligner        represented on the analysis image, and/or for an image attribute        of the analysis image, the image attribute relating to a        separation between at least one tooth represented on the        analysis image, and the aligner represented on said analysis        image;    -   c″) preferably, assessment of the suitability of the aligner as        a function of the value of said tooth or image attribute;    -   d″) preferably, sending of an information message as a function        of said assessment.

As will be seen in more detail hereinafter in the description, anassessment method according to the invention considerably simplifies theassessment of the good suitability of the aligner to the treatment,while making this assessment particularly reliable. In particular, themethod may be implemented from simple photographs or films, taken withno particular precautions, for example by the patient. The number ofappointments with the orthodontist may therefore be limited.

Preferably, in the step b″), all said analysis tooth zones areidentified, and the value of said tooth attribute is determined for eachanalysis tooth zone, and, in the step c″), the suitability of thealigner is determined as a function of said tooth attribute values.

Preferably, said tooth attribute is chosen from the group formed by amaximal separation along the free edge of the tooth, a mean separationalong the free edge of the tooth, and said image attribute is chosenfrom the group formed by a maximal separation along all of the teethrepresented, a mean separation along the free edges of all of the teethrepresented, an overall acceptability of the separation of the teethrepresented.

The tooth attribute relating to a separation may in particular be theexistence of a separation, this attribute being able to take the toothattribute values “yes” or “no”; or a value measuring the scale of theseparation, for example an observed maximal separation or an assessmentin relation to a scale.

In the step b″), a detailed analysis method according to the inventionis preferably implemented, a tooth attribute of each historical toothzone of each historical image of the learning base relating to aseparation between the tooth represented by the historical tooth zone,and an aligner worn by said tooth and represented on said historicalimage.

Preferably, the step b″) comprises the following steps:

-   -   b″1) preferably before the step a″), creation of a learning base        comprising more than 1000, preferably more than 5000, preferably        more than 10 000 images of dental arches, or “historical        images”, each historical image representing an aligner worn by a        “historical” patient and comprising one or more zones each        representing a tooth, or “historical tooth zones”, to each of        which is assigned, for at least one tooth attribute relating to        a separation between the tooth represented by the historical        tooth zone considered, and the aligner represented, a tooth        attribute value;    -   b″2) learning of a deep learning device, preferably a neural        network, by means of the learning base;    -   b″3) submission of the analysis image to the deep learning        device for the deep learning device to determine at least one        probability relating to        -   the presence, at a location of said analysis image, of an            analysis tooth zone, and        -   the tooth attribute value of the tooth represented on said            analysis tooth zone;    -   b″4) determination, as a function of said probability, of the        presence of a separation between the aligner and the tooth        represented by said analysis tooth zone, and/or of an amplitude        of said separation.

The steps b″1) to b″4) may comprise one or more of the features,possibly optional, of the steps 1) to 4) described above, respectively.

In one embodiment, in the step b″), a global analysis method accordingto the invention is implemented, an image attribute of each historicalimage of the learning base relating to a separation between at least onetooth represented on the historical image, and an aligner worn by saidtooth and represented on said historical image.

Preferably, the step b″) comprises the following steps:

-   -   b″1′) creation of a learning base comprising more than 1000        images of dental arches, or “historical images”, each historical        image comprising an attribute value for at least one image        attribute, or “image attribute value”, relating to a separation        between at least one tooth represented on the analysis image,        and the aligner represented on said analysis image;    -   b″2′) training of at least one deep learning device, preferably        a neural network, by means of the learning base;    -   b″3′) submission of the analysis image to the deep learning        device for it to determine, for said analysis image, at least        one probability relating to said image attribute value, and        determination, as a function of said probability, of the        presence of a separation between the aligner and the tooth or        teeth represented on the analysis image, and/or of an amplitude        of said separation.

The steps b″1′) to b″3′) may comprise one or more of the features,possibly optional, of the steps 1′) to 3′) described above,respectively.

The method is now described when a detailed analysis is implemented inthe step b″).

Prior to the step a″), the learning base must be enriched, preferablyaccording to an enrichment method according to the invention, in orderto contain historical images whose description specifies, for each ofthe historical tooth zones, a value for the tooth attribute relating tothe separation.

This information may be input manually. For example, an imagerepresenting one or more so-called “historical” tooth zones may bepresented to an operator, preferably an orthodontist, and he or she maybe asked to identify these historical tooth zones and indicate, for eachhistorical tooth zone, whether there is a separation or not and/orassess the amplitude of this separation.

A historical image may be a photo representing an aligner worn by ahistorical patient. Alternatively, a historical image may be the resultof a processing of an image representing a naked dental arch (that is tosay without aligner) and of an image representing the same arch bearingthe aligner. The image representing the naked arch may in particular bea view of a model of the arch deformed to obtain a maximal match withthe image representing the arch bearing the aligner. Such a processingmay in particular be useful for better revealing the outline of theteeth and of the aligner when the teeth are not very visible through thealigner.

In the step a″), the acquisition of the analysis image may be performedlike the acquisition of the updated images in the step B) describedabove.

Preferably, at least one reminder informing the patient of the need tocreate an analysis image is addressed to the patient. This reminder maybe in paper form or, preferably, in electronic form, for example in theform of an email, of an automatic alert from a dedicated mobile deviceor an SMS. Such a reminder may be sent by the orthodontic practice orlaboratory or by the dentist or by the dedicated mobile application ofthe patient, for example.

The step a″) is performed at the moment when the assessment of the shapeof an aligner is desired, for example more than 1 week after the startof the treatment with the aligner.

The analysis image is an image representing the aligner worn by theteeth of the patient.

In the step b″), the analysis image is analyzed according to a detailedanalysis method according to the invention.

The deep learning device has been trained by means of a learning basecontaining historical images whose description specifies, for at leastone, preferably each historical tooth zone, a value for a toothattribute relating to a separation between the tooth represented by thehistorical tooth zone and the aligner worn by said tooth and representedon said historical image.

The value for this tooth attribute therefore provides an informationitem relating to the shape of the aligner relative to the shape of theteeth of the patient.

The value for this tooth attribute can be a measurement of theseparation, for example a measurement of the maximum separation, or ofthe mean separation for the tooth represented by the historical toothzone.

The deep learning device is therefore capable of analyzing the analysisimage to determine, preferably for each of the “analysis tooth zones”,the existence, even the extent, of a separation of the aligner of thetooth represented on the analysis tooth zone.

In the step c″), an assessment is made, as a function of the results ofthe preceding step, as to the compatibility the aligner. For example, asearch is conducted to see if the separation of the aligner with atleast one tooth exceeds an acceptability threshold and, in this case, adecision is made as to the replacement of the aligner by a better suitedaligner.

The suitability of the aligner can be assessed in the context of anorthodontic treatment (separation compatible or not with the orthodontictreatment) but also in the context of a non-therapeutic, in particularesthetic, treatment. Aligners can indeed be used to displace teeth forpurely esthetic purposes, without this displacement modifying the stateof health of the patient. The suitability of the aligner can also beassessed in the context of a research program concerning theeffectiveness of the aligner, for example to assess a new material forthe aligner, on a human being or on another animal

In the step d″), an information item relating to the assessment in thepreceding step is sent, in particular to the patient and/or theorthodontist.

The orthodontist can then use this information, possibly in combinationwith additional information, for example the age of the patient or theduration for which the aligner has been worn, to establish a diagnosisand, if necessary, decide on a suitable treatment.

In one embodiment, the method comprises, in the step b″), a globalanalysis according to the invention. The other steps are unchanged.

The analysis of the analysis image and of the historical images is thenperformed globally, without identification of the individual situationof each of the teeth represented and the image attribute relates to theimage as a whole.

For example, the image attribute relating to the separation may relateto the acceptability of a dental situation, the cause of one or moreseparations, or relate to the global scale of the separation orseparations of the teeth. For example, the value of the image attributemay be “globally acceptable” or “globally unacceptable”. The value ofthe image attribute can also, for example, be a measurement of theseparation, for example a measurement of the maximum separation, or ofthe mean separation between the teeth represented on the analysis imageand the aligner.

As now clearly emerges, a method according to the invention makes itpossible, from simple photos or a simple film, to determine whether thealigner is abnormally detached, even, if a detailed analysis wasperformed in the step b″), to determine the regions in which the aligneris separated from the teeth and assess the scale of this separation.

The invention also relates to a method for adapting an orthodontictreatment, a method in which a method for assessing the shape of anorthodontic aligner according to the invention is implemented, then, asa function of the result of said assessment, a new aligner ismanufactured and/or the patient is counseled, for example for him or herto improve the conditions of use of his or her orthodontic aligner, inparticular the positioning and/or the time bands when it should be wornand/or the maintenance of his or her orthodontic aligner, in order tooptimize the treatment.

The use of aligners is not limited to therapeutic treatments. Inparticular, an assessment method may be implemented to assess an alignerexclusively used for esthetic purposes.

The method may also be used to assess other dental, in particularorthodontic, parts or appliances.

Computer Program

The invention relates also:

-   -   to a computer program, and in particular a dedicated application        for cellphones, comprising program code instructions for the        execution of one or more steps of any method according to the        invention, when said program is run by a computer,    -   to a computer medium on which such a program is stored, for        example a memory or a CD-ROM.

Obviously, the invention is not limited to the embodiments describedabove and represented.

In particular, the patient is not necessarily a human being. A methodaccording to the invention may be used for another animal.

The patient could be alive or dead. He is preferably alive.

The methods of the invention may be used for an orthodontic treatment,but also out of any orthodontic treatment, and even out of anytherapeutic treatment.

1. A method for analyzing an image, called “analysis image”, of a dentalarch of a patient, method in which the analysis image is submitted to aneural network, in order to determine at least one value of an imageattribute relating to the analysis image, the analysis image being aphotograph or an image taken from a film, in which said image attributerelates to a position or an orientation or a calibration of anacquisition apparatus used to acquire said analysis image or acombination thereof, or a quality of the analysis image, and inparticular relating to the brightness, to the contrast or to thesharpness of the analysis image, or a combination thereof.
 2. The methodas claimed in claim 1, said method comprising the following steps: 1′)creation of a learning base comprising more than 1000 images of dentalarches, or “historical images”, each historical image comprising anattribute value for the at least one image attribute, or “imageattribute value”; 2′) training of the neural network, by means of thelearning base; 3′) submission of the analysis image to the neuralnetwork for it to determine, for said analysis image, at least oneprobability relating to said image attribute value, and determination,as a function of said probability, of a value for said image attributefor the analysis image.
 3. The method as claimed claim 2, in which thelearning base comprises more than 10 000 historical images.
 4. Themethod as claimed in claim 1, in which the analysis image is acquired bya cellphone.
 5. The method as claimed in claim 1, in which theacquisition of the analysis image is performed by the patient.
 6. Themethod as claimed in claim 1, the analysis image being analyzed in realtime.
 7. The method as claimed in claim 1, said method comprising thefollowing steps: a′) activation by the patient of a cellphone so as toacquire the analysis image; b′) analysis of the analysis image accordingto the method as claimed in claim 1, so as to determine the at least onevalue of an image attribute relating to the analysis image.
 8. Themethod as claimed in claim 7, the method comprising following step d′):d′) comparison of said image attribute value with an instruction.
 9. Themethod as claimed in claim 8, the method comprising following step e′):e′) sending of an information message as a function of said comparison.10. The method as claimed in claim 9, in which the message guides anoperator performing the acquisition to acquire the analysis image. 11.The method as claimed in claim 1, the analysis image representingseveral teeth.
 12. The method as claimed in claim 7, in which severalanalysis images are acquired in succession.
 13. The method as claimed inclaim 12, in which the analysis images are acquired from differentacquisition directions.
 14. The method as claimed in claim 13, in whichat least analysis images from the front, from the right and from theleft of the patient are acquired.
 15. The method as claimed in claim 7,the analysis image being analyzed in real time.
 16. The method asclaimed in claim 15, in which the image acquisition apparatus alerts anoperator performing the acquisition in order for the operator to modifythe acquisition direction.
 17. The method as claimed in claim 1, inwhich the analysis image is acquired for establishing a remotediagnosis.
 18. A method for analyzing an image, called “analysis image”,of a dental arch of a patient, said method comprising : the submissionof the analysis image to a neural network, in order to determine atleast one value of a tooth attribute relating to the analysis image, theanalysis image being a photograph or an image taken from a film, and thedetermination of a value for an image attribute as a function of saidtooth attribute value, said image attribute relating to a position or anorientation or a calibration of an acquisition apparatus used to acquiresaid analysis image or a combination thereof, or a quality of theanalysis image, and in particular relating to the brightness, to thecontrast or to the sharpness of the analysis image, or a combinationthereof.
 19. The method as claimed in claim 18, said method comprisingthe following steps: a′) activation of an image acquisition apparatus soas to acquire the analysis image; b′) analysis of the analysis image bymeans of a deep learning device trained by means of a learning base, soas to identify at least one analysis tooth zone representing a tooth onsaid analysis image, and to determine at least one tooth attribute valuefor said analysis tooth zone, as claimed in claim 18; c′) determination,for the analysis image, of a value for an image attribute, said valuebeing a function of said tooth attribute value, as claimed in claim 18.20. The method as claimed in claim 18, in which the analysis image isacquired by a cellphone.
 21. The method as claimed in claim 18, in whichthe acquisition of the analysis image is performed by the patient. 22.The method as claimed in claim 19, the step b′) comprising the followingsteps: 1) creation of a learning base comprising more than 1000 imagesof dental arches, or “historical images”, each historical imagecomprising one or more zones each representing a tooth, or “historicaltooth zones”, to each of which, for the at least one tooth attribute, atooth attribute value is assigned; 2) training of at least one deeplearning device by means of the learning base; 3) submission of theanalysis image to said at least one deep learning device for it todetermine at least one probability relating to the presence, at alocation in said analysis image, of an analysis tooth zone,representing, at least partially, at least one tooth in the analysisimage, the tooth attribute value of the at least one tooth representedon the tooth zone; 4) determination, as a function of said probability,of the presence of a tooth of said arch at a position represented bysaid analysis tooth zone, and of the attribute value of said tooth. 23.The method as claimed in claim 19, the method comprising following stepd′): d′) comparison of said image attribute value with an instruction.24. The method as claimed in claim 23, the method comprising followingstep e′): e′) sending of an information message as a function of saidcomparison.
 25. The method as claimed in claim 22, in which at step 1),the learning base is composed according to the following steps: A) at an“updated” instant, production of a model of a dental arch of a patient,or “updated reference model”, and segmentation of the updated referencemodel so as to produce, for each tooth, a “tooth model”, and, for atleast one tooth attribute, assignment of a tooth attribute value to eachtooth model; B) acquisition of at least one image of said arch, or“updated image”, in real acquisition conditions; C) for each updatedimage, search for virtual acquisition conditions suitable for anacquisition of an image of the updated reference model, called“reference image”, exhibiting a maximal match with the updated image insaid virtual acquisition conditions, and acquisition of said referenceimage; D) identification, in the reference image, of at least one zonerepresenting a tooth model, or “reference tooth zone” and, by comparisonof the updated image and the reference image, determination, in theupdated image, of a zone representing said tooth model, or “updatedtooth zone”; E) assignment, to said updated tooth zone, of the toothattribute value or values of said tooth model; F) addition of theupdated image enriched with a description of said updated tooth zone andits tooth attribute value or values, or “historical image”, in thelearning base.
 26. The method as claimed in claim 22, in which at step1), the learning base is composed according to the following steps: A′)at an initial instant, production of a model of a dental arch of apatient, or “initial reference model”, and segmentation of the initialreference model so as to produce, for each tooth, a “tooth model”, and,for at least one tooth attribute, assignment of a tooth attribute valueto each tooth model; B′) at an updated instant, acquisition of at leastone image of said arch, or “updated image”, in real acquisitionconditions; C′) search, by deformation of the initial reference model,for an updated reference model and virtual acquisition conditionssuitable for an acquisition of an image of the updated reference model,called “reference image”, exhibiting a maximal match with the updatedimage in said virtual acquisition conditions; D) identification, in thereference image, of at least one zone representing a tooth model, or“reference tooth zone” and, by comparison of the updated image and thereference image, determination, in the updated image, of a zonerepresenting said tooth model, or “updated tooth zone”; E) assignment,to said updated tooth zone, of the tooth attribute value or values ofsaid tooth model; F) addition of the updated image enriched with adescription of said updated tooth zone and its tooth attribute value orvalues, or “historical image”, in the second learning base.
 27. Themethod as claimed in claim 24, in which the message guides an operatorperforming the acquisition to acquire the analysis image.
 28. The methodas claimed in claim 18, the analysis image representing several teeth.29. The method as claimed in claim 19, in which several analysis imagesare acquired in succession.
 30. The method as claimed in claim 29, inwhich the analysis images are acquired from different acquisitiondirections.
 31. The method as claimed in claim 30, in which at leastanalysis images from the front, from the right and from the left of thepatient are acquired.
 32. The method as claimed in claim 19, theanalysis image being analyzed in real time.
 33. The method as claimed inclaim 32, in which the image acquisition apparatus alerts an operatorperforming the acquisition in order for the operator to modify theacquisition direction.
 34. The method as claimed in claim 18, in whichthe analysis image is acquired for establishing a remote diagnosis. 35.The method as claimed in claim 18, in which the tooth attribute ischosen from a tooth number, a tooth type, a tooth shape parameter, atooth appearance parameter, a parameter relating to the state of thetooth, an age for the patient, or a combination of these attributes.